Adaptive Observers for MIMO Discrete-Time LTI Systems

In this paper, an adaptive observer is proposed for multi-input multi-output (MIMO) discrete-time linear time-invariant (LTI) systems. Unlike existing MIMO adaptive observer designs, the proposed approach is applicable to LTI systems in their general form. Further, the proposed method uses recursive least square (RLS) with covariance resetting for adaptation that is shown to guarantee that the estimates are bounded, irrespective of any excitation condition, even in the presence of a vanishing perturbation term in the error used for updation in RLS. Detailed analysis for convergence and boundedness has been provided along with simulation results for illustrating the performance of the developed theory.Comment: 6 pages, 5 figure

Safe reinforcement learning control for continuous-time nonlinear systems without a backup controller

This paper proposes an on-policy reinforcement learning (RL) control algorithm that solves the optimal regulation problem for a class of uncertain continuous-time nonlinear systems under user-defined state constraints. We formulate the safe RL problem as the minimization of the Hamiltonian subject to a constraint on the time-derivative of a barrier Lyapunov function (BLF). We subsequently use the analytical solution of the optimization problem to modify the Actor-Critic-Identifier architecture to learn the optimal control policy safely. The proposed method does not require the presence of external backup controllers, and the RL policy ensures safety for the entire duration. The efficacy of the proposed controller is demonstrated on a class of Euler-Lagrange systems

Safe Q-learning for continuous-time linear systems

Q-learning is a promising method for solving optimal control problems for uncertain systems without the explicit need for system identification. However, approaches for continuous-time Q-learning have limited provable safety guarantees, which restrict their applicability to real-time safety-critical systems. This paper proposes a safe Q-learning algorithm for partially unknown linear time-invariant systems to solve the linear quadratic regulator problem with user-defined state constraints. We frame the safe Q-learning problem as a constrained optimal control problem using reciprocal control barrier functions and show that such an extension provides a safety-assured control policy. To the best of our knowledge, Q-learning for continuous-time systems with state constraints has not yet been reported in the literature

Adaptive Output Feedback Model Predictive Control

Model predictive control (MPC) for uncertain systems in the presence of hard constraints on state and input is a non-trivial problem, and the challenge is increased manyfold in the absence of state measurements. In this paper, we propose an adaptive output feedback MPC technique, based on a novel combination of an adaptive observer and robust MPC, for single-input single-output discrete-time linear time-invariant systems. At each time instant, the adaptive observer provides estimates of the states and the system parameters that are then leveraged in the MPC optimization routine while robustly accounting for the estimation errors. The solution to the optimization problem results in a homothetic tube where the state estimate trajectory lies. The true state evolves inside a larger outer tube obtained by augmenting a set, invariant to the state estimation error, around the homothetic tube sections. The proof for recursive feasibility for the proposed `homothetic and invariant' two-tube approach is provided, along with simulation results on an academic system.Comment: 6 page

Modeling and parametric optimization of 3D tendon-sheath actuator system for upper limb soft exosuit

This paper presents an analysis of parametric characterization of a motor driven tendon-sheath actuator system for use in upper limb augmentation for applications such as rehabilitation, therapy, and industrial automation. The double tendon sheath system, which uses two sets of cables (agonist and antagonist side) guided through a sheath, is considered to produce smooth and natural-looking movements of the arm. The exoskeleton is equipped with a single motor capable of controlling both the flexion and extension motions. One of the key challenges in the implementation of a double tendon sheath system is the possibility of slack in the tendon, which can impact the overall performance of the system. To address this issue, a robust mathematical model is developed and a comprehensive parametric study is carried out to determine the most effective strategies for overcoming the problem of slack and improving the transmission. The study suggests that incorporating a series spring into the system's tendon leads to a universally applicable design, eliminating the need for individual customization. The results also show that the slack in the tendon can be effectively controlled by changing the pretension, spring constant, and size and geometry of spool mounted on the axle of motor

Decentralized nonlinear adaptive optimal control scheme for enhancement of power system stability

An adaptive decentralized control scheme is proposed for real-time control of oscillatory dynamics and overall stability improvement of an interconnected power system. A standard framework of continuous-time (CT) infinite-horizon optimal control paradigm is defined and an extended online actor-critic (AC) algorithm based on policy iteration is used for its solution. The AC structure uses neural networks (NNs) whose weights are updated adaptively. The proof for the convergence of the scheme guarantees the system stability. The unobservable internal states of the synchronous machine are estimated using a numerically stable, swift and relatively accurate decentralized dynamic state estimator (DDSE) based on spherical-radial cubature rule. Applicability of the developed scheme has been demonstrated on a benchmark power system (IEEE 2.2, 16 machine 68 bus system) model via nonlinear time-domain simulations. Multi-processor technology based scaled laboratory setup was used for controller performance validation in real-time